Shared-counter image sensor

ABSTRACT

An image sensor generates first digital samples and second digital samples during respective first and second sampling intervals, the first digital samples including at least one digital sample of each pixel of a first plurality of pixels, and the second digital samples including at least one digital sample of each pixel of a second plurality of pixels. A sum of the first digital samples is accumulated within a first counter as the first sampling interval transpires, and a sum of the second digital samples is accumulated within the first counter as the second sampling interval transpires.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/686,697, filed Apr. 14, 2015 and entitled “Shared-Counter ImageSensor” (now U.S. Pat. No. 9,826,176), which is a continuation of U.S.patent application Ser. No. 13/864,427, filed Apr. 17, 2013 and entitled“Shared-Counter Image Sensor” (now U.S. Pat. No. 9,036,065) which claimspriority to (i) U.S. Provisional Application No. 61/684,117, filed Aug.16, 2012 and entitled “Binary Pixel Image Sensor Sampled At Non-UniformIntervals” and (ii) U.S. Provisional Application No. 61/700,119, filedSep. 12, 2012 and entitled “Shared-Counter Image Sensor.” Each of theforegoing patent applications is hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to the fields of electronic image sensorsand digital image processing.

BACKGROUND

Despite ubiquitous application, conventional CMOS image sensors sufferfrom a number of limitations. First, the linear relationship betweenphoton strikes and pixel value yields a relatively small dynamic rangein which a pixel quickly reaches saturation under brighteningconditions. Additionally, because the maximum number of detectablephoton strikes is proportional to pixel size, the pixel footprint isdictated by the dynamic range required in a given application and doesnot scale with shrinking process geometries. In high-end digitalcameras, like DSLRs (digital single-lens reflex) for example, the photodiode tends to be four or more micrometers at each edge in order toachieve a reasonable dynamic range, consuming an area hundreds or eventhousands of times the minimum transistor size permitted by leadinglogic process geometries.

BRIEF DESCRIPTION OF THE DRAWINGS

The various embodiments disclosed herein are illustrated by way ofexample, and not by way of limitation, in the figures of theaccompanying drawings and in which like reference numerals refer tosimilar elements and in which:

FIG. 1 illustrates an embodiment of an integrated-circuit image sensor(image sensor IC) having a binary-pixel image sensor together withcircuitry to manage sensor operations and communicate with a hostintegrated circuit;

FIGS. 2A-2C illustrate embodiments of conditional-reset sense amplifiersand corresponding binary pixels that may be employed to achieve variabletemporal oversampling within a binary pixel image sensor;

FIG. 3 is a timing diagram illustrating an exemplary operation of asense-amp super pixel that includes the conditional-reset senseamplifier and binary pixel of FIG. 2A;

FIG. 4A illustrates an embodiment of a conditional-reset sense-amp superpixel that applies multiple reference voltages to respective subsets ofbinary pixels to effect spatially-distributed, non-uniform thresholds;

FIG. 4B illustrates an exemplary read-out sequence that may be executedwith respect to the conditional-reset sense-amp super pixel of FIG. 4A;

FIG. 5 illustrates an embodiment of a multi-thresholding,conditional-reset sense amplifier that may be employed within thesense-amp super pixel (SSP) of FIG. 4A;

FIG. 6A illustrates an embodiment of a reference generator that may beused to generate the reference voltages supplied to the multi-thresholdSSP of FIG. 4A and applied within the sense amplifier of FIG. 5;

FIG. 6B illustrates an exemplary threshold profile that may be generatedby the reference voltage generator of FIG. 6A or a variation thereof;

FIG. 6C illustrates an exemplary threshold profile that may be effectedby applying reference voltages generated by the reference voltagegenerator of FIG. 6A (or a variation thereof) as pixel reset voltages;

FIG. 6D illustrates an exemplary threshold profile effected by applyingmultiple pixel reset voltages in combination with multiple thresholdvoltages;

FIG. 7 illustrates an embodiment of a conditional-reset sense-amp superpixel having variably-sized binary pixels to effect aspatial-distribution of non-uniform thresholds without requiringmultiple reference voltages;

FIG. 8 illustrates an exemplary sequential application of differentreference voltages to the binary pixels of an image sensor to effect atemporal rather than a spatial distribution of different samplingthresholds;

FIG. 9 illustrates an embodiment of a threshold generator that may beused to generate sequentially adjusted sampling thresholds as describedin reference to FIG. 8;

FIGS. 10A and 10B illustrate operation of a conditional-reset binarypixel sensor in low-light and high-light conditions;

FIG. 11 illustrates an approach for extending the dynamic range of aconditional-reset binary pixel;

FIG. 12A illustrates a sensor control logic embodiment that may be usedto implement the control logic of FIG. 1 in a binary pixel image sensorhaving non-uniform sampling intervals;

FIG. 12B illustrates an exemplary flow of the finite state machine ofFIG. 12A;

FIG. 13 illustrates exemplary non-uniform sampling interval progressionsthat may be employed to balance the pixel sampling workload through anexposure interval;

FIG. 14 illustrates an exemplary image reconstruction circuit (e.g.,implemented on chip with the binary pixel image sensor or in a separateIC die) that may be used to generate image pixels in response to rawalgorithmic super pixel (ASP) values obtained from binary pixels sampledwith non-uniform sampling intervals;

FIG. 15A illustrates an alternative embodiment of a conditional-resetsense amplifier together with a binary pixel that permits correlateddouble-sampling;

FIG. 15B is a timing diagram illustrating an exemplary operation of asense-amp super pixel that includes the conditional-reset senseamplifier and binary pixel of FIG. 15A;

FIG. 16 illustrates an alternative embodiment of a binary pixel imagesensor having a shared-counter architecture;

FIG. 17 illustrates a spatial/temporal oversampling sequence withrespect to an image pixel of the image pixel mat detailed in FIG. 16;

FIG. 18 illustrates a sequence of pixel-count transfer and counter resetoperations that may be executed within the image sensor of FIG. 16, withthe output of a pixel mat counter being transferred to memory and thecounter reset after each ‘n’ successive spatial oversampling sequenceswith respect to a given image pixel;

FIG. 19 illustrates an exemplary pixel sampling sequence in which eachimage pixel within a pixel mat is sampled with three different samplingintervals over the course of an exposure time, including a number offine sampling intervals and progressively extended first and secondcoarse sampling intervals;

FIG. 20 illustrates an exemplary data-transfer sequence effected bytime-staggering the sample-sequencing of image pixel mats that share acommon resource to align data transfer bursts from one pixel mat with anon-burst interval of another;

FIG. 21 illustrates an exemplary storage of fractional image pixel countvalues within the memory component of FIG. 16.

FIG. 22 illustrates an alternative image sensor architecture in whichtwo rows of image pixel mats are operated in parallel, with each pair ofco-operated image pixel mats forming a double mat;

FIG. 23 illustrates a stacked-die implementation of an image sensor thatmay be used to implement the generalized image sensors of FIGS. 1 and16;

FIG. 24 illustrates the different pixel clock and transmit clock domainsdescribed in reference to FIG. 1; and

FIG. 25 illustrates an image system architecture that may be used toimplement the image sensor, image data storage and image reconstructionfunctions described above in reference to FIG. 1.

DETAILED DESCRIPTION

In various digital image sensor embodiments disclosed herein, pixels areclustered together with an embedded sense amplifier and sensed withrespect to a binary threshold to yield a collection of single-bit pixelvalues that can be combined to form the output of an image pixel.Because integrated-circuit (IC) process geometries permit pixel sizesbelow the diffraction limit of visible light, a photon striking within aSDL cluster of such “binary” pixels may activate (i.e., be detected byand exceed the threshold of) one binary pixel but not others illuminatedby the same blurred source, thus decreasing the number of non-activatedpixels available to detect subsequent photons from the source.Consequently, when exposed to a stationary photon flux, the rate ofpixel activation slows as the exposure interval transpires (i.e., due tophotons striking already activated pixels), thereby yielding alogarithmic rather than linear sensitivity profile and thus apotentially much higher dynamic range than achieved through prior-arttechniques (in other embodiments disclosed herein this behavior ismodified while still improving dynamic range). Further, because thepixels can be scaled with decreasing process geometries, pixel densitiescan increase freely with process improvement, thus overcoming physicalimpediments faced by conventional architectures and enablingdramatically higher performance in future generations of imagingdevices. In other embodiments, counter circuits for accumulating pixelsample values are shared among multiple groups of pixels, thus enablingsample data compression and increased temporal oversampling without animpractically large number of counter circuits. These and otheradvantages, features and embodiments are described below.

FIG. 1 illustrates an embodiment of a sensor IC 100 having abinary-pixel image sensor 101 together with circuitry to manage sensoroperations and communicate with a host IC. More specifically, a physicalsignaling interface 109 (PHY) is provided to receive commands andconfiguration information from a host IC (e.g., a general-purpose orspecial-purpose processor, application-specific integrated circuit(ASIC) or any other control component) and control logic 103 is providedto execute incoming commands within an operating configuration set inresponse to the configuration information. For purposes of example, thesensor IC and host IC are assumed to form the primary image acquisitioncomponents within a camera (e.g., a still-image or video camera within amobile device, compact camera, digital SLR camera, stand-alone orplatform-integrated webcam, high-definition video camera, securitycamera, automotive camera etc.). The sensor IC and host IC can be moregenerally deployed alone or together with like or different imagingcomponents within virtually any imaging system or device includingwithout limitation metrology instruments, medical instruments, gamingsystems or other consumer electronics devices, military and industrialimaging systems, transportation-related systems, space-based imagingsystems and so forth.

Continuing with FIG. 1, control logic 103 issues control and clocksignals to row control logic 105 and output logic 107. As shown, the rowcontrol logic includes a row counter 112 to sequence through the rows ofSSPs 111 within the image sensor during image acquisition and read-outoperations, and a sense-amp controller 114 to control sensing forrespective SSPs during those operations. Accumulator logic 107 receivespixel sample values via bit lines 106 and accumulates sample valuesobtained from respective pixels (or SSPs) into respective image datavalues, for example, by summing or otherwise combining temporallyoversampled outputs from individual pixels or SSPs. PHY 109 serializesdata read-out from image sensor 101 for transmission to a memorycomponent, host IC and/or other processing component via a relativelynarrow external signal path. This operation is discussed in greaterdetail below.

In the embodiment of FIG. 1, binary-pixel image sensor 101 is composedof N rows and M columns of sense-amp super pixels (SSPs), with each SSPincluding an embedded sense amplifier and P binary pixels. Row counter112 outputs N sets of P row signals, with each set of P row signalscoupled to the binary pixels that populate a logical or physical row ofSSPs. More specifically, each signal within the set of P row signals iscoupled to a respective one of the P binary pixels within each SSP ofthe row, thereby enabling M like-positioned binary pixels withinrespective SSPs of a given row to be selected as a group duringprecharge and sense operations. The logical collection of binary pixelsselected in response to assertion of a given one of the row signals isreferred to herein as a “binary pixel row” to distinguish it from therows of SSPs themselves (which each may contain one or more binary pixelrows). The individual row signals are referred to accordingly asbinary-pixel-row (BPR) signals with each set of P BPR signals beingindexed by a respective SSP row index ‘n’ that ranges from 0 to N−1, andeach BPR signal within a given set of P BPR signals being furtherindexed by a binary pixel index, ‘p’ that ranges from 0 to P−1.

In one embodiment, row control logic 105 includes state logic thatsequences between states to control the operation of row counter 112 andsense amp controller 114, for example, incrementing row counter 112 toselect the next binary pixel row in an incremental progression afterenabling sense amp controller 114 to complete a sequence of operationswith respect to the binary pixels of the currently indexed binary pixelrow. The sense-amp controller itself outputs respective sets of senseamp control signals (SACtrl[N−1:0]) to the rows of SSPs with each set ofsense-amp control signals including constituent signals to enable theembedded sense amplifiers within a selected SSP row to be operated inparallel, in one embodiment for example, sequencing as a unit throughprecharge, sense and read-out states as detailed below.

Reflecting on the image sensor architecture shown in FIG. 1, it can beseen that a significant number of conductors can overlay the imagesensor to instantiate the BPR lines, sense-amp control lines and corebit lines shown. In one embodiment, the backside of the die on which theimage sensor is formed is ground (or lapped or otherwise thinned) to athickness that permits pixel exposure through the backside of the die,thereby enabling accumulation of light unimpeded by the image sensorwiring. In such an embodiment, the front-side of the ground die can beadhered to a substrate or to the die of another IC (e.g., a host IC) toprovide mechanical stability.

In the embodiment of FIG. 1, each sense-amp super pixel 111 includespixel reset logic that conditionally resets individual pixels of the SSPaccording to their sample values. More specifically, if the sampleobtained from a given pixel indicates that the binary threshold has beenexceeded (i.e., pixel saturated), the pixel is reset (i.e., photodiodeor other light-detecting element precharged) to enable further photondetection. Otherwise, if the sample indicates that the binary thresholdhas not been exceeded, the pixel state is left unchanged (i.e., no resetevent). Through this conditional-reset operation, individual pixelswithin the SSP and the pixel array as a whole are reset at varying timesthroughout an exposure interval (i.e., recycling the well of thephotodiode or other light-detecting element to enable further photonstrike detection), and are thus temporally oversampled at an effectiverate that varies according to the number of pixel resets within a givenexposure interval.

Each conditional-reset SSP receives binary pixel row signals(BPR[n][P−1:0]) and sense amp control signals (SACntrl[n]) correspondingto an ‘nth” SSP row within image sensor 101 and, when enabled, outputspixel data onto a core bit line 106 to be counted within accumulatorlogic 107 for eventual output via PHY 109. In the exemplary embodimentshown, conditional-reset SSP 111 includes an embedded sense amplifier125 coupled to sixteen binary pixels through one or more sample lines(“sample”) and one or more pixel reset lines (“pxrst”). Though shown atthe margin of SSP 111, sense amplifier 125 may alternatively bedistributed among binary pixels 123. As in all embodiments, more orfewer binary pixels 123 may be present than those shown.

The sense amp control signals (SACntrl) provided to embedded senseamplifier 125 include, in addition to other signals discussed below, adigital or analog threshold signal (“thresh”) and a reset-enable signal(“rst_en”). The threshold and reset-enable signals are applied withinthe sense amplifier as shown conceptually at 126. That is, comparatorcircuitry 127 compares a threshold (conveyed or selected by thethreshold signal) with a binary pixel value driven onto the sample line(i.e., by a binary pixel selected by assertion of a corresponding BPRsignal) to produce a pixel data signal (“data”) which, in turn, is ANDedwith the reset-enable signal in logic 129 to drive the pixel reset line.By this operation, a reset signal is asserted on the pixel reset line toreset the selected binary pixel only if that binary pixel has receivedsufficient light to yield an over-threshold binary pixel value. Becausethe conditional pixel reset events are synchronized by the timing of thereset-enable signal (itself asserted, for example, at regular intervalswithin a sequence of pixel cycles), the effective exposure interval fora given binary pixel is established at a predetermined number of clockedexposure intervals.

Referring to detail view 130 of a binary pixel 123, the pixel reset lineis coupled to pixel reset logic 131 where it is logically ANDed with apixel-specific control signal (e.g., the binary pixel row signal for thesubject pixel, though a different control signal may alternatively beused) to selectively switch on reset transistor 133 and thereby couplethe detection node of the pixel (“VDET”) to a precharge voltage source.Accordingly, when the pixel reset signal is asserted by the senseamplifier (i.e., in response to a binary pixel value that exceeds thesampling threshold), and the pixel-specific control signal is alsoasserted, the binary pixel is reset. Through this operation, each of thebinary pixels within conditional-reset SSP 111 may be reset at differenttimes and thus accumulate light without reset over disparate numbers ofclocked exposure intervals in accordance with their respective photonabsorptions.

FIGS. 2A-2C illustrate embodiments of conditional-reset sense amplifiersand corresponding binary pixels that may be employed to achieve variabletemporal oversampling within a binary pixel image sensor. FIG. 2Aillustrates a current-mode sense amplifier 170 can be employed, togetherwith multiple instances of binary pixel 185, to form a sense-amp superpixel. Instead of employing back-to-back inverters (as may be done in analternative embodiment), the sense amplifier is formed by a differentialamplifier. Thus, no latching operation occurs, avoiding the need for asense amp precharge operation. Also, the amplification transistor 132within binary pixel 185 operates as the input transistor of thedifferential pair, in effect distributing a portion of the differentialamplifier within the binary pixels of the SSP (in alternativeembodiments, a dedicated input transistor may be provided within thesense amplifier as a counterpart to transistor 171).

Due to the potentially imbalanced legs of the differential amplifier(i.e., one leg effected through the binary pixel via sample andcurrent-source lines (“sample” and “csrc”) and thus through a longersignal path than the other leg, which is confined within the locale ofthe sense amplifier), transistors 173 and 174 are coupled in a currentmirror configuration to establish matching current sources for the twolegs. Accordingly, when a bias voltage is applied to transistor 187,powering the sense amplifier, a differential voltage is developed ondifferential nodes 186 a, 186 b according to whether the pixel voltage(i.e., the voltage at the detection node of photosensitive element 134)is greater or less than a threshold voltage applied to the gate oftransistor 171. More specifically, complement-data node 186 b drops to alower potential than counterpart node 186 a if the pixel voltage isbelow the threshold (i.e., transistor 132 will exhibit a lowertransconductance (higher resistance) than transistor 171 due to thedifference in their gate voltages and thus effect a higher IR drop thantransistor 171) and conversely rising to a higher potential than node186 a if the pixel voltage is above the threshold. An inverter formed bytransistors 176 and 177 amplifies the potential on complement-data node186 b, outputting a data signal representative of the binary pixel state(e.g., to a core bit line).

Sense amplifier 170 additionally includes conditional-reset circuitry toassert an active-low pixel-reset signal if the binary pixel valueexceeds (i.e., detection node voltage is lower than) the samplingthreshold, thus enabling a conditional reset operation within theattached pixel. More specifically, the data output of the senseamplifier and a reset-enable signal (“rst_en”) are supplied to a logicNAND circuit formed by transistors 178, 179, 180 and 181, switching ontransistors 178 and 179 and switching off transistors 180 and 181 topull the pixel reset line (“/pxrst”) low if the sense amp data output(generated by inverter transistors 176 and 177) and reset-enable signalsare both high. Conversely, the NAND circuit pulls the pixel reset linehigh (to a deasserted state) via transistor 181 or 180 (and switchingoff at least one of transistors 178 and 179) if either of the dataoutput or the reset-enable signal are low.

Within binary pixel 185, the active-low state of the pixel reset lineswitches on logic AND transistor 191, thereby passing the state of thebinary pixel row signal to the gate of reset transistor 187.Accordingly, reset transistor 187 is switched on to reset (i.e.,precharge) the binary pixel when the pixel-reset line is pulled low andthe binary pixel row signal is asserted. Altogether, when the binarypixel row signal is raised to enable the binary pixel value to besampled (i.e., enabling the charge level of the detection node onto thesample line) and the binary pixel value exceeds the sampling thresholdsuch that a logic ‘1’ is generated at the sense amp data output, then aconcurrent assertion of the reset-enable signal will, in conjunctionwith the over-threshold binary pixel value, pull the pixel-reset linelow to effect a pixel reset operation within the binary pixel selectedby the asserted binary pixel row signal. When the pixel reset signal ishigh, pull-down transistor 190 is switched on within binary pixel 185 toground the gate of reset transistor 187 and thus prevent resetregardless of the state of the binary pixel row signal.

As noted above, the reset-enable signal and threshold value constitute asubset of the sense-amp control signals provided to the sense amplifierwithin a given sense-amp super pixel. In the embodiment of FIG. 2A, thesense-amp control signals additionally include a clamp signal that isasserted between pixel sampling events to switch on transistor 175,coupling the output node of the differential amplifier (i.e., drain oftransistor 171) to ground and thus avoiding a floating input to theinverter formed by transistors 176 and 177 (preventing, among otherthings, a metastable output on the data line). The bias signal isapplied to the gate of current-sinking transistor 172 to effect adesired amplification level within the sense amplifier. In oneembodiment, the bias signal is an analog voltage generated, for example,by a current mirror or other bias-control circuit to establish (inconjunction with current-mirror coupled load transistors 173, 174) adesired bias current within the differential amplifier. In analternative embodiment, the bias signal is a multi-bit digital signalthat is applied to a bank of parallel transistors (representedsymbolically by transistor 172) so that, as the number of logic ‘1’ bitswithin the bias signal is increased or decreased, a corresponding numberof transistors within the parallel bank are switched on to establish acorresponding bias current. In either implementation, calibrationoperations may be performed at system startup and/or occasionallythereafter to establish/maintain an appropriate bias signal setting.

FIG. 2B illustrates an alternative embodiment of aconditional-pixel-reset sense amplifier 195 and conditionally-resettablebinary pixel 205. Sense amplifier 195 works similarly to the senseamplifier of FIG. 2A, except that transistors 196, 197, 198 and 199 areprovided to logically AND an active-low reset-enable signal (/rst_en)and the complement data signal output from the differential amplifierpair (i.e., from the drain of transistor 171). That is, when thereset-enable signal is lowered, transistor 199 is switched on to powerthe inverter formed by transistors 196 and 198. Consequently, when thecomplement data signal is low (i.e., the binary pixel output exceeds thesampling threshold), the inverter output goes high (i.e., transistor 198is switched on, while transistors 196 and 197 are switched off), therebydriving the pixel reset line high to effect a pixel reset.

In contrast to the embodiment of FIG. 2A, the pixel reset logic withinbinary pixel 205 is formed by only two transistors (not three), but iscontrolled by a pixel-row reset signal (BPR_reset) that is distinct fromthe binary pixel row signal used to enable the detection node voltage tobe sensed (i.e., “BBR_sense” in the embodiment shown). Morespecifically, the AND transistor 191 and pull-down transistor 190 shownin FIG. 2A are omitted in favor of an additional reset transistor 206coupled between sense-amp-controlled reset transistor 207 (whichcorresponds to transistor 187 in the embodiment of FIG. 2A) and thedetection node. The two reset transistors 206, 207 are coupled to thepixel-row reset input (BPR_reset) and the pixel reset line (pxrst),respectively, so that, when both inputs are high, the detection node ofthe binary pixel is switchably coupled to a precharge voltage source(e.g., V_(DD)) to reset the binary pixel.

Still referring to FIG. 2B, in an alternative embodiment a single binarypixel row signal may be supplied to the gates of access transistor 130and reset transistor 206, provided that any charge trapped in thesource-to-drain coupling between the two reset transistors 206 and 207(i.e., trapped by virtue of pixel reset signal assertion in connectionwith other binary pixels of the SSP) does not intolerably disturb thepixel sampling result.

FIG. 2C illustrates another alternative embodiment of aconditional-pixel-reset sense amplifier 215 and conditionally-resettablebinary pixel 220. In this case, the sense amplifier works as a voltagemode amplifier instead of a current mode amplifier by virtue of localamplifying transistor 216. Because the differential amplifier sensingcurrent passes through amplifying transistor 216 instead of transistor132 within binary pixel 220, transistor 132 may be tied high as shownand thus operated as a follower amplifier (providing an output to thegate of amplifying transistor 216 via access transistor 130) thatfollows the state of the detection node. While a follower amplifier isshown, in all such in-pixel amplifying arrangements, the gain of theamplifying transistor or amplifying circuit may be greater than, lessthan or equal to one. Also, while the conditional pixel reset circuitrywithin the binary pixel and sense amplifier corresponds to that shown inFIG. 2A, the approaches described in reference to FIG. 2B can beemployed in alternative embodiments.

FIG. 3 is a timing diagram illustrating an exemplary operation of an SSPthat includes the conditional-reset sense amplifier and binary pixel ofFIG. 2A. Operations within the SSP are executed synchronously withrespect to a clock signal (clk) and are shown as a sequence of shadedevents (224-233) in respective clock cycles. Initially, at 224, a forcedreset is executed with respect to the binary pixel by asserting(raising) the clamp, reset (rst_en) and binary pixel row signals whiledeasserting the bias signal, thus forcing a low output from thedifferential amplifier to emulate an over-threshold condition andtrigger a pixel reset. That is, referring briefly to FIG. 2A, the highdata output of inverter 176/177 is NANDed with the reset-enable signalto drive the pixel-reset line low, which, in combination with theasserted binary pixel row (bpr) signal, produces a high signal at thegate of the reset transistor 187 to reset the binary pixel (charge orpre-charge the detection node).

Following the pixel reset at 224, the binary pixel is exposed over alight accumulation interval. Note that the light accumulation intervalmay span many clock cycles as explained above and thus the detectionnode voltage is shown as having a steady declining slope—in actuality,the detection node voltage will decrease stepwise in response tonon-uniformly spaced photon strikes. At 225, the clamp signal is loweredwhile the bias and binary pixel row signals are raised, therebyinitiating a pixel sense/read-out operation. Because the detection nodevoltage has dropped to a level below the threshold voltage (shown as asteady-state signal level superimposed over the detection node voltage),the differential amplifier generates a logic low output that is invertedto form a logic ‘1’ binary pixel sample. In the embodiment shown, thebinary pixel sample is output (“data”) onto a core bit line (i.e., to becounted by an on-chip or off-chip ASP accumulator) over a pair of clockcycles before the bias and binary pixel row signals are lowered (and theclamp signal raised) to conclude the sense/read-out operation at 227. Inthe second clock cycle of the read-out interval, after the binary pixelsample value has stabilized at the sense amp data output, a conditionalreset operation is executed as shown at 226. More specifically, thereset signal is raised in response to the rising clock edge as shown,thus causing the NAND gate formed by transistors 178, 179, 180, 181 inFIG. 2A to assert or deassert an active-low pixel reset signal (drivingthe pixel reset line low or high, respectively) according to whether anover-threshold condition was detected with respect to the subject binarypixel. Accordingly, the logic ‘1’ (over-threshold) binary pixel samplein the example shown triggers assertion of the active-low pixel resetsignal, which, in combination with the continued assertion of the binarypixel row signal, effects a pixel reset.

The sense/read-out and conditional-reset operations are repeated at 228,229 and 230 following a second light accumulation interval, but a lesserlight accumulation yields an under-threshold detection node voltage(i.e., a detection node voltage that does not exceed—that is, go lowerthan—the threshold level). Consequently, the resulting logic ‘0’ binarypixel sample yields a logic high (unasserted) pixel reset signal andthus no pixel reset occurs. Accordingly, at the conclusion of thesense/read-out at interval 230, the binary pixel is again allowed toaccumulate light over an exposure interval, with the partiallydischarged detection node voltage produced in the preceding exposureinterval being carried forward as the initial condition for thesubsequent exposure interval. Consequently, even though the lower photonflux (i.e., lower than in the exposure preceding the sense operation at225) continues, the non-reset during conditional reset interval 229effectively joins the two clocked exposure intervals (i.e., the exposureinterval between 227 and 228, and the exposure interval between 230 and231) into a single effective exposure interval such that the detectionnode voltage at the subsequent sense/read-out interval 231 reflects thetotal number of photon strikes during those two clocked exposureintervals. In the example shown, the photon count over the last twoclocked exposure intervals is sufficient to exceed the threshold andthus yields a logic ‘1’ data output that contributes to the net ASPvalue and produces the pixel reset shown at 232. Though a photonaccumulation without reset over two exposure intervals is shown, lowerlevels of photon flux may result in continuous photon accumulation overany number of clocked exposure intervals between one and the total imageframe interval (or other maximum) before a forced reset is executed.

Mathematical analysis and simulation results indicate the potential forundesired regions of zero or near zero slope in the response curve of avariable temporal oversampling image sensor at low luminance due to thequantized nature of the temporal oversampling. In a number ofembodiments, those errors and resulting output anomalies are mitigatedor avoided altogether by effecting non-uniform sampling thresholdswithin the sensor. In a first set of non-uniform threshold embodiments,for example, different reference voltages are applied to respectivebinary pixels that contribute to a single image pixel to effect aspatial distribution of non-uniform thresholds. That is, each of thepixels is sampled with respect to a given reference voltage (thresholdvoltage) and thus yields a single-bit digital sample, but the referencevoltage applied in at least one of the binary pixels differssubstantially from the reference voltage applied in at least one otherof the binary pixels during a given exposure interval, so that a rangeof reference voltages is applied to respective binary pixels. In anotherset of non-uniform threshold embodiments, binary pixels of varying sizes(i.e., exhibiting non-uniform light accumulation areas) are providedwithin each image pixel field to effect a spatial-distribution ofnon-uniform thresholds with a single reference voltage, thus avoidingthe complexities and potential errors involved with generating multiplereference voltages. In yet another set of non-uniform thresholdembodiments, different reference voltages are applied sequentially tothe binary pixels of the image sensor in respective portions of theimage frame period to effect a temporal rather than a spatialdistribution of thresholds, thereby achieving the benefits of multiplereference voltages without the added wiring and in-situ referenceselection circuitry required for spatial distribution of the referencevoltages. Instead of or in conjunction with any of the non-uniformthreshold embodiments, non-uniform thresholds can also be effected byvarying the reset voltage applied to one or more of the binary pixels,either spatially or temporally. By intentionally “under-precharging” abinary pixel to a voltage that is closer to the comparison reference,the threshold is effectively lowered because fewer photon strikes arerequired to discharge the pixel to a voltage below the referencevoltage.

FIG. 4A illustrates an embodiment of a conditional-reset SSP 245 thatapplies multiple reference voltages to respective subsets of binarypixels to effect spatially-distributed, non-uniform thresholds. Ingeneral, multiple sampling thresholds between q_(min) (≥single-photoncharge dissipation) and q_(max) are determined by expected incidentluminances, spatial and temporal oversampling factors and physicalconstraints on VLSI design and manufacture and applied to respectivegroups of binary pixels 247 within the SSP and thus within an imagesensor as a whole. In a number of embodiments, the thresholddistribution is determined so as to optimize a luminance fidelitymetric, (e.g., minimize the signal to noise weighted by the expectedprobability of finding a given luminance level). In at least one suchdetermination, for instance, all other parameters being equal, thedistribution is dominated by low-threshold values in low-luminanceconditions and by high-threshold values in high-luminance conditions.Once the number of binary pixels associated with each threshold value,‘q’, has been determined, the spatial arrangement of all the binarypixels may be determined so as reduce spatial artifacts in the finalrendered image, in particular to reduce aliasing artifacts. In oneembodiment, for example, the spatial distribution of different-thresholdbinary pixels is chosen so as to minimize the peak of thetwo-dimensional Fourier transform of the image pixel sensitivity.

Still referring to FIG. 4A, SSP 245 operates generally as explainedabove in reference to FIG. 1, receiving a set of binary pixel rowsignals (BPR[n][15:0] in this example) and sense-amp control signals(SACntrl[n]), and executing conditional reset operations to effectvariable temporal oversampling. Instead of receiving a single reference(threshold) voltage, however, the embedded sense amplifier 249 receivesmultiple reference voltages, depicted in detail view 250 as q0, q1, q2,q3 (or q[3:0]), and a select signal (“sel”) that indicates which of thereference voltages is to be applied to establish the threshold for agiven binary pixel sense/read-out operation. In the embodiment shown,sixteen binary pixels 247 are split into four threshold-groups with eachof the four threshold groups being sensed in comparison to a respectiveone of the four thresholds, q0, q1, q2 or q3, as indicated within thelabel of the binary pixel. Further, the binary pixels 247 thatconstitute each of the four groups are spatially dispersed (orscattered) within the SSP such that each of the thresholds is applied toan expansive rather than a concentrated region within the SSP. In FIG.4A, for example, the footprint of each subgroup extends to upper, lower,right and left edges of the binary pixel area; a region that can beenvisioned, for example, by a box outline that includes each of theshaded binary pixels to which threshold q0 is applied.

In one embodiment, illustrated by the exemplary SSP read-out sequence inFIG. 4B, the binary pixel pixels of SSP 245 are read-out in a sub-grouporder that yields a stepwise increase in the applied threshold. That is,the binary pixels of sub-group 0 (BP0-BP3) are read out in the first setof four pixel sense/read-out operations, while the threshold selectvalue is set to select reference voltage q0 as the applied threshold(“applied thresh”), a selection effected by selector element 255 asshown within detail view 250 of sense amplifier 249. After thesense/read-out of the binary pixels of sub-group 0 (each of whichincludes a comparison with the selected q0 threshold in comparatorcircuitry 251 to yield a data value that is ANDed with a reset-enablesignal in logic 253 to yield a conditional pixel reset signal) iscompleted, the binary pixels of sub-group 1 (i.e., BP4-BP7) are read outwhile reference voltage q1 is selected as the applied threshold, thenthe binary pixels of sub-group 2, and finally the binary pixels ofsub-group 3. In alternative embodiments, the sub-groups may be sensedand read-out in an order reverse of that shown (i.e., ramping theapplied threshold down from q3 to q0) or in a scattered order in whichthe threshold select signal transitions as frequently as the binarypixel row signal. As the application of a given threshold in a binarypixel sense/read-out operation defines the subject binary pixel as beingpart of the threshold subgroup, the various different subgroup read-outsequences may be used to establish a desired distribution of binarypixel sub-groups within the SSP; distributions that may be programmablydetermined (e.g., by one or more fields within a threshold policyregister) and thus changed in accordance with application requirementsor dynamically according to ambient or other conditions.

FIG. 5 illustrates an embodiment of a multi-thresholding,conditional-reset sense amplifier 265 that may be employed within theSSP of FIG. 4A. In general, sense amplifier 265 operates as discussed inreference to FIG. 2C or 2A except that input transistor 171 of thosesense amplifiers is replaced (or supplemented) by multi-thresholdcircuitry 267 (“mt”). In one implementation, shown for example in detailview 270, multi-threshold circuitry 267 includes an input transistor 271corresponding to transistor 171 of FIG. 2C/2A together with amultiplexer (formed by pass gates 273 ₀-273 ₃) that applies one of thefour incoming reference voltages (q[3:0] in this example) to the gate ofinput transistor 271 in accordance with the state of the thresholdselect signal, “sel.” In alternative embodiments, single-transistorpass-gates (or other switching elements) may be used instead of thetwo-transistor pass gates shown, and more or fewer reference voltagesmay be provided.

Still referring to FIG. 5, an alternative embodiment of themulti-threshold reference circuitry includes parallel reference paths asshown in detail view 280. Each of the reference paths includes an inputtransistor (281 a, 282 a, 283 a or 284 a) coupled to receive arespective one of reference voltages q[0]-q[3] (again, there may be moreor fewer reference voltages than the four shown), and an enabletransistor (281 b, 282 b, 283 b or 284 b) coupled to receive arespective bit of the threshold select signal. By this arrangement, thethreshold select signal may be output in one of four one-hot states(i.e., one bit set, the others cleared) to couple the input transistorof a selected one of the reference paths between transistors 174 and 172of sense amplifier 265, thereby establishing the corresponding referencevoltage as the applied threshold. Again, there may be more or fewerreference voltages (and corresponding reference paths) than the fourshown. Also, an additional enable transistor may be disposed in seriesbetween transistors 173 and 172 of the sense amplifier and/or otherloading element coupled to the drain terminal of bias transistor 172 forload matching purposes.

FIG. 6A illustrates an embodiment of a reference generator 290 that maybe used to generate the reference voltages supplied to themulti-threshold SSP of FIG. 4A and applied within the sense amplifier ofFIG. 5. As shown, reference generator 290 includes a base register 291to store a programmed or predetermined base reference value, “q-base,”and a set of canonically coupled multipliers 293, 295, 297. The outputof base register 291 and each multiplier 293, 295, 297 is supplied to arespective digital-to-analog converter (292, 294, 296, 298), therebyyielding a set of reference voltages having an amplitudeq[i]=q-base^(i)*^(M), where i is the index of the reference voltage andM is the factor applied within each multiplier 293, 295, 297. In oneembodiment, for example, M=2 so that each reference voltage is twicethat of the nearest reference (and so that each multiplier may beimplemented by a small-footprint shift arrangement that prepends a ‘0’bit in the least significant bit position of the incoming referencevalue to generate an output reference value). Other multiplicationfactors may be applied by the multipliers 293, 295, 297 in alternativeembodiments, including non-uniform multiplication factors (e.g.,applying M₁, M₂ and M₃, where M₁≠M₂ and/or M₂≠M₃). More generally, anynumber of reference voltages may be generated with any practicablevoltage distribution, including distributions that are determinedadaptively or heuristically instead of according to predeterminedformulae. In all such cases, the reference voltages may be supplied toan adjuster circuit 299 that calibrates and/or adapts the referencevoltages according to their differences or other source of informationindicating a deviation from desired thresholds. Also, base register 291may be loaded with an updated value in a register programming operation,or incremented/decremented as part of a calibration loop or adaptiveloop (e.g., receiving an increment/decrement signal from adjuster 299)to shift all the reference voltages up or down.

In one implementation, the digital-to-analog converters of FIG. 6Agenerate threshold voltages as respective offsets from a pixel reset (orprecharge) voltage, thus yielding the threshold profile shown in FIG.6B. In another embodiment, the reference voltages output from generator290 are applied as pixel reset voltages instead of decision thresholds,thus enabling binary pixel sub-groups to be reset to respective voltageshaving different offsets from a threshold level. FIG. 6C illustrates anexample of such an arrangement, with the outputs q[3:0] of the referencegenerator in FIG. 6A constituting binary pixel reset voltagesV_(RST)[3:0]. As shown, the different reset voltages effect differentbinary pixel sampling thresholds without requiring multiple referencevoltages to be delivered to the sense amplifier (i.e., the output of allbinary pixels may be compared with a single reference voltage to yield alogic ‘1’ or logic ‘0’ binary pixel value). Though exponentially relatedthreshold voltages and reset voltages are shown in FIGS. 6B and 6C,numerous other voltage step sizes (including adaptively oralgorithmically determined step sizes as well as programmably controlledstep sizes) may be applied in alternative embodiments. Also, the twoapproaches shown in FIGS. 6B and 6C (multi-thresholding throughapplication of non-uniform threshold voltages and multi-thresholdingthrough application of non-uniform pixel reset voltages) may be combinedas shown in FIG. 6D. In the example shown, two different pixel resetvoltages (VRST[1:0]) are applied in combination with two differentreference voltages (q[1:0]) to achieve four distinct binary pixelthresholds. Though a linear threshold step (n, 2n, 3n, 4n) is shown inFIG. 6D, other threshold steps may be effected in alternativeembodiments.

FIG. 7 illustrates an embodiment of a conditional-reset SSP 305 havingvariably-sized (i.e., non-uniform) binary pixels 307 to effect aspatial-distribution of non-uniform thresholds without requiringmultiple reference voltages. In the example shown, the SSP includes fivebinary pixels, including two “1×” binary pixels, BP0 and BP1, havingsizes corresponding to singe binary pixel cells, a “2×” binary pixel(BP2) sized to occupy approximately twice the area of a 1× binary pixel,a 4× binary pixel (BP3) sized to occupy approximately four times thearea of a 1× binary pixel and an 8× binary pixel (BP4) sized to occupyapproximately eight times the area of a 1× binary pixel. In oneembodiment, the 1× binary pixels and the 4× binary pixels have a squareaspect, while the 2× and 8× binary pixels have an oblong aspect, thusenabling all the binary pixels to be disposed compactly within a squareSSP footprint. Different size ratios and binary pixel aspect ratios maybe used in alternative embodiments (e.g., each binary pixel may have anaspect ratio of 1/2^(1/2) so that when doubled along the shorterdimension to produce a larger binary pixel, the same aspect ratio ismaintained), particularly where the SSP footprint is oblong ornon-quadrilateral.

Regardless of the exact pixel size ratios and aspect ratios, all elsebeing equal, larger binary pixels can have effectively lower thresholds(greater sensitivity) than smaller binary pixels when their outputs arecompared with a consistent reference voltage. This is particularly truewith extremely small pixel sizes (i.e., at sizes where the capacitanceof the photodiode itself does not dominate the overall capacitance ofthe sensing node) as the larger of two such pixels receives on averageproportionally more photoelectrons, and therefore exhibits a fasterdecline in detection node voltage (on average), than the smaller of twosuch pixels for the same incoming photon flux. Accordingly, theeffective threshold, “t_(eff),” for the 2×, 4× and 8× binary pixels canapproach ½, ¼ and ⅛ that of the 1× binary pixel for the same referencevoltage, ‘q’ and thus are shown as ‘q/2’, ‘q/4’ and ‘q/8’, respectively.In devices where the photodiode capacitance is non-negligible, a lesspronounced difference will be observed—the differences can becharacterized for a specific implementation and expressed as a set ofthresholds.

Still referring to FIG. 7, the SSP receives binary pixel row signals andsense amp control signals generally as described in reference to FIG. 13(including a single reference voltage (‘q’) and a reset-enable signal(“rst_en”) to time the variable temporal oversampling operationdescribed above), except that the number of binary pixel row signals isreduced to account for the reduced number of binary pixels per unit area(i.e., a logarithmically reduced number of binary pixels in theembodiment shown). The reference voltage, q, is applied to all of thebinary pixels, thus obviating the distribution and selection of multiplereference voltages. In one embodiment, the same binary pixel layout isapplied in all SSPs. In alternative embodiments, the sizes and/orrelative positioning of differently-sized binary pixels may be alteredfrom ASP to ASP (e.g., mirrored across an axis). Also, though asingle-reference voltage embodiment is shown, multiple referencevoltages may be used in combination with non-uniform binary pixel sizesto effect multi-thresholding in other embodiments. For example, tworeference voltages can be applied in combination with two sizes ofbinary pixels to achieve four effective thresholds within a given SSP.

FIG. 8 illustrates an exemplary sequential application of differentreference voltages to the binary pixels of an image sensor to effect atemporal rather than a spatial distribution of thresholds. In theexample shown, the T clocked exposure intervals (each corresponding to aread-out of the image sensor) that constitute an image frame period aredivided into a number of sub-frame intervals (five in the exampleshown), each associated with a respective sampling threshold. Thus, asingle threshold ‘q’ is applied in sense/read-out operations executedwithin all binary pixels within the image sensor for the exposureintervals that constitute a first sub-frame interval, S1. At theconclusion of S1, the threshold is adjusted (e.g., doubled in thisexample) and again applied in sensor-wide binary pixel sense/read-outoperations for the exposure intervals that constitute a second sub-frameinterval, S2. This threshold-adjust and sensor read-out approach isrepeated for each of the remaining sub-frame intervals, until theconclusion of the image frame interval is reached (i.e., image samplecount=T). In alternative embodiments the threshold voltage may bestepped between more, fewer and/or different threshold levels than thoseshown, and the threshold steps may progress downward instead of upwardor even be non-monotonic. Also, the number of image samples acquired forrespective thresholds may be non-uniform (i.e., capturing more imagesamples at one threshold than another). Further, all such parameters(threshold value applied in each sub-frame interval, number of imagesamples per sub-frame interval, number of threshold steps per imageframe) may be varied dynamically according to ambient conditions orother considerations (e.g., power mode, image resolution, ISO, etc.).The temporally adjusted thresholds may be employed in combination witheither or both of the spatially-distributed threshold approachesdescribed above (i.e., different reference voltages applied inconnection with respective sub-groups of binary pixels and/orvariably-sized binary pixels). Moreover, in all such cases, conditionalreset may be employed so that a given binary pixel is reset only if itsbinary pixel value exceeds the threshold applied in the samplinginterval.

FIG. 9 illustrates an embodiment of a threshold generator 320 that maybe used to generate sequentially adjusted sampling thresholds asdescribed in reference to FIG. 8. In one embodiment, threshold generator320 is included within the row control logic of an image sensor IC(e.g., logic 155 of FIG. 1) and outputs a sequence of reference voltagesas part of the sense-amp control signals supplied to the SSPs of theimage sensor. In alternative embodiments, the threshold generator may bedisposed elsewhere within an image sensor IC, or even partly orcompletely off-chip (e.g., in a host processor that outputs a sequenceof digital reference values to be converted to analog form within theimage sensor IC and applied in binary pixel sense/read-out operations).

In the embodiment shown, reference generator 320 includes a modulo Tcounter 321, threshold select logic 329, sub-frame-interval register325, threshold register 323, D/A converter bank 327 and selector circuit331. Subframe-interval register 325 includes storage fields to storesub-frame count values above which a corresponding one of the thresholdsprogrammed within threshold register 323 is to be applied. In theimplementation shown, for example, sub-frame-interval register 325includes four storage fields that define the durations (in clockedexposure intervals) of the final four sub-frame intervals within theimage frame period, with the first sub-frame interval (i.e., S1) beingimplied by the S2 sub-frame count value. More or fewer sub-frameintervals than the five shown may be supported in alternativeembodiments.

Still referring to FIG. 9, modulo-T counter 321 counts transitions of aclock signal, Clk, counting up from zero to T−1 before overflowing tozero (or down from T−1 to zero before underflowing to T−1). The counteroutput, which represents a count of the sensor read-out being performed(i.e., the ‘i^(th)’ one of the T image samples to be acquired within theimage frame period) and thus a sample count, is supplied together withthe sub-frame count values from register 325 to threshold select logic329. Threshold select logic 329 compares the sample count to thesub-frame count values to produce a threshold-select value 330corresponding to the sub-frame count range in which the sample countfalls. That is, as shown in conceptual view 335, if the sample count isless than the S2 sub-frame count value (negative determination at 339),threshold select logic 329 outputs a threshold-select value to selectorcircuit 331 to select the q0 reference voltage (i.e., voltage levelgenerated by D/A converter bank 327 in response to the q0 valueprogrammed within threshold register 323) to be output as the sub-framethreshold, ‘qs’ (340). After the sample count reaches the S2 sub-framecount (affirmative determination at 339) the threshold select logicoutputs a threshold-select value to select the q1 reference voltage at342 and continues to do so until the sample count reaches the S3sub-frame count (affirmative determination at 341). Upon reaching the S3sub-frame count, the threshold select logic outputs a threshold-selectvalue to select the q2 reference voltage at 344 and continues to do sountil the sample count reaches the S4 sub-frame count (affirmativedetermination at 343). Similarly, upon reaching the S4 sub-frame count,the threshold select logic outputs a threshold-select value to selectthe q3 reference voltage at 346 and continues to do so until the samplecount reaches the S5 sub-frame count (affirmative determination at 345),after which the threshold select logic outputs a threshold-select valueto select the q4 reference voltage at 348.

The threshold generator of FIG. 9 may be varied in numerous ways inalternative embodiments. For example the D/A converters may be omittedand the threshold values instead output as set of digital signals (e.g.,an N-bit signal that can be used within recipient sense amplifiers toadjust a digitally-controlled reference). Also, a single “base”threshold value may be programmed within a threshold control registerand applied within other circuitry to derive the remaining thresholds(or all of the thresholds) as in the embodiment of FIG. 6A. Similarly, asingle sub-frame count value (or smaller number of sub-frame countvalues) may be programmed and used to derive the remaining sub-framecount values. More generally, any circuitry capable of outputtingdifferent thresholds for different sub-frame intervals within an imageframe interval may be used in alternative embodiments.

Numerous techniques may be applied to calibrate thresholds in amulti-threshold binary pixel image sensor including, for example andwithout limitation, image-data-dependent calibration techniques as wellas precision reference comparison. In all cases, calibration operationsmay be executed by logic on the image sensor die and/or on an IC coupledto the image sensor IC. Calibration operations may be carried out atregular intervals, opportunistically (e.g., when an idle state isdetected or another maintenance operation is being performed) or inresponse to events such as threshold divergence detection, user input,etc. Also, in some embodiments, the calibration techniques may be usednot only to adjust the sensor thresholds, but also (or alternatively) todetect the ratios of the observed thresholds and adjust the imagereconstruction algorithms to reconstruct relative luminance based on theobserved thresholds from the instant the image was gathered.

Reconstruction of a multilevel-per-pixel image from a variable-temporaloversampled—and potentially multi-thresholded—binary pixel array issomewhat different than for a single-threshold and/or uniformlytemporally oversampled pixel array. For a single-threshold array, eachASP can return a value ACC(i,j) for the number of binary ‘1’ pixeloutputs observed spatially and temporally within that ASP for an imageframe. ACC(i,j) can be accumulated on-chip, partially on-chip andpartially off-chip, or completely off-chip, e.g., using the methodsdescribed above for resetting the binary pixel array. Once theaccumulation is made, however, the device completing the accumulationcan either convert the accumulation to a relative luminance value orpass the raw accumulation downstream to a storage or processing devicefor downstream conversion.

The relative luminance value estimated for a conditionally-reset binarypixel ASP is, in one embodiment, the maximum likelihood relativeluminance mapping to ACC(i,j). Due primarily to photon shot noise andquantization noise bias (there is a non-zero likelihood that a binarypixel will exceed the current threshold by more than one photoelectronbetween sample times), a range of luminance can produce each possiblevalue of ACC(i,j). The maximum likelihood relative luminance is theluminance value that is most probable given ACC(i,j). The values can bederived from an equation and stored in a lookup table or evaluateddirectly by a processor, derived via simulation and stored in a lookuptable, or derived via controlled measurements and stored in a lookuptable. A lookup table can be stored in on- or off-chip nonvolatilememory, transferred from such memory to an on- or off-chip volatilememory for use, or created directly by a host processor for a givencurrent thresholds/samples setting set.

A variety of approaches exist when the conditionally-reset binary pixelASP is operated in a multi-threshold mode. In one approach, a valueACC(i,j) is returned for each ASP as in the single-threshold case, andevaluated by an equation or input to a lookup table as in the caseabove. The equation/lookup table result depends on the actual thresholdsused and the number of sample points available per ASP at each samplepoint. When the equation or lookup table inaccurately models thesefactors, reconstruction errors will result. The calibration methodsdescribed above can reduce such errors in an appropriate embodiment.

Although a direct estimate from a multi-threshold joint-accumulated ASPis possible, such an estimate is generally sub-optimal. Optimality isnot achieved because not all accumulated samples have the sameprobability density function (PDF)—intuitively, a threshold that isexceeded at almost every observation point has a much larger uncertaintythan one that is exceeded at roughly every third observation point. Thusa better estimate can be obtained if it is possible for the binarysensor to return, for each ASP, separate accumulations ACC(q_(n))(i,j)for each threshold q_(n). A relative luminance maximum likelihoodfunction can then be evaluated from the joint probability densityfunction of all ACC(q_(n))(i,j), considered together. In one embodiment,the joint PDF is stored as a multidimensional lookup table, from whichrelative luminance is interpolated. In another embodiment, the PDFs areassumed independent and are represented separately for each thresholdq_(n). The individual PDFs are then jointly evaluated to produce anestimate.

FIGS. 10A and 10B illustrate operation of a conditional-reset binarypixel sensor in low-light and high-light conditions, respectively. Inthe low-light case (FIG. 10A), the total number of photon strikes overthe duration of the exposure interval (or frame interval, t_(FRAME)) isinsufficient to exceed the binary detection threshold (q) so that apixel value of zero (no detection) results. In a number of embodiments,the pixel control logic may detect this zero-detect condition andresponsively adjust the detection threshold (e.g., iteratively reducingthe number of photon strikes required to yield a ‘1’), thus adaptivelyincreasing low-light sensitivity.

In FIG. 10B, the high luminance fills the photodiode well (i.e., binarypixel saturates) during every sampling interval, thus yielding a logic‘1’ sample (and triggering a reset) for each of the N samples. That is,the high light intensity has exceeded the dynamic range of the pixel(i.e., with the response curve flat-lining at the maximum sample value)such that further increase in intensity (or even decrease down to thepoint at which at least one logic ‘0’ is detected) is undetected.

FIG. 11 illustrates an approach for extending the dynamic range of aconditional-reset binary pixel. As shown, instead of sampling the binarypixel at uniform intervals throughout the frame period, the frame periodis divided into a number of non-uniform sampling intervals in which thelongest sampling interval (τ_(max)) is longer than a sampling intervalin the uniform case (τ_(fix)) and the shortest sampling interval(τ_(min)) is shorter than the uniform sampling interval. In oneembodiment, for example, a logarithmic progression of sampling intervalsfrom longest to shortest (or vice-versa, though non-monotonicprogressions may also be used) is applied, with the sum of n_(t)sampling intervals totaling to the frame period (i.e., t_(exp) ort_(FRAME)) and the sampling interval durations defined as follows:

${\frac{\tau_{i}}{\tau_{m\; i\; n}} = {{round}\lbrack 2^{\frac{i - 1}{n_{t} - 1}{\log_{2}{(\frac{\tau_{{ma}\; x}}{\tau_{m\; i\; n}})}}} \rbrack}},$where t_(i) is the duration of the i^(th) sampling interval (i rangingfrom 0 to n_(t)−1). Other interval-duration progressions may be used inalternative embodiments, including linear progressions, heuristicallydetermined progressions, user-specified patterns, etc.

As shown in FIG. 11, the shortened sampling intervals permit detectionof non-activated binary pixels (i.e., sampling a logic ‘0’) even in thehigh-light condition that saturated the uniform-progression binary pixelsensor. That is, by shortening the time between a subset of the sampleswithin the frame period, it becomes possible to distinguish betweenintensity variations even at extremely high-light conditions, thusextending the dynamic range of the sensor. More generally varying theduration of the temporal sampling periods may enable the same dynamicrange to be achieved as in the uniform sampling-period case in fewertotal samples (i.e., same dynamic range at reduced bit depth) and thuswith reduced power consumption. Alternatively (or additionally), varyingthe duration of the temporal sampling periods may enable improveddynamic range and signal-to-noise ratio with only one or a few binarythresholds, thus simplifying sensor implementation.

FIG. 12A illustrates an embodiment of sensor control logic 420 that maybe used to implement control logic 153 of FIG. 3 in a binary pixel imagesensor having non-uniform sampling intervals. As shown, a configurationregister 421 may be programmed with parameters including, for exampleand without limitation, frame period (τ_(exp)), maximum samplinginterval duration (τ_(max)), minimum sampling interval duration(τ_(min)), number of samples (n_(t)), progression policy (e.g., linear,logarithmic, heuristic, prescribed pattern, etc.) and so forth. Thecontents of the programmable register form a tuple (i.e., a compositevalue) that is applied to lookup table 423 to select an entry containinga set of sampling intervals and their respective numbers of occurrenceswithin each frame period. A finite state machine 425 steps through(i.e., selects in turn) each sampling interval (τ)/occurrence-count (Q)pair within the selected table entry to generate row control/timingsignals and read-out control/timing signals that correspond to thevarious sampling intervals and their corresponding numbers ofoccurrences.

FIG. 12B illustrates an exemplary flow of the finite state machine 425(FSM) of FIG. 12A. Starting at 427, the variable sampling interval (τ)and occurrence count (Q) are initialized to a first interval/occurrencepair (τ₁, Q₁) obtained from the interval lookup table (i.e., element 423of FIG. 24A), and an occurrence index, i and interval index j are set toinitial values (i.e., i=1, j=1). At 428, the FSM allows light to beaccumulated within the binary pixel array for duration τ, followed byeither a destructive-sampling operation with a hard reset, or anon-destructive sampling operation with a conditional reset. Theoccurrence index is incremented at 429 and then compared with theoccurrence count (Q) at 430. If the occurrence index does not exceed theoccurrence count (i.e., negative determination at 430), the lightaccumulation operation at 428 is repeated to acquire another samplefollowing light accumulation at the same sampling interval. After thefinal occurrence of a given sampling interval (i.e., affirmativedetermination at 430), the interval index T is incremented at 431 andcompared with the maximum index (i.e., n_(t), the total number ofsampling intervals per exposure interval) at 432. If the interval indexdoes not exceed the maximum index, then the occurrence index is resetand a new sampling interval and occurrence count (t_(j), Q_(j)) areassigned as the sampling interval and occurrence count for the nextiteration of the operations at 428-430. If the interval index exceedsthe maximum index, then the exposure interval is deemed to be complete.

Depending on light level, a quality setting, or some other parameter, itmay be desirable to adjust the shortest sampling intervals to a durationthat makes it impossible to sample all rows in the array at thatduration during a single pass. In such a case, different parts of thearray can be sequenced differently to accommodate the shortest sampling.For instance, some of the rows of the array can be in a long-durationsample interval while others are in shorter-duration sampling intervals,with the overall scheduling adjusted to not exceed the sample ratelimitations of the pixel readout (or device readout) circuitry. Oneexample of such non-uniform sampling interval progressions is presentedin FIG. 13. As shown, the sampling interval progressively shortens forrow ‘j’ (e.g., following a logarithmic, linear, or other progression),and progressively lengthens for row ‘k,’ thereby balancing the pixelsampling workload during the exposure interval shown (t_(exp)). Whilesymmetric progressions are shown (i.e., the Δτ progression in row ‘k’ isthe mirror image of the At progression in row ‘j’), numerous otherprogression interval variations may be employed in alternativeembodiments.

FIG. 14 illustrates an exemplary image reconstruction circuit (e.g.,implemented on chip with the binary pixel image sensor or in a separateIC die) that may be used to generate image pixels in response to raw ASPvalues obtained from binary pixels sampled with non-uniform samplingintervals. In the particular embodiment shown, a set of lookup tables435 ₀-435 _(N-1) each corresponding to a response curve (e.g., as shownin FIG. 31) for a given pattern of non-uniform sampling intervals,thresholds and other parameters expressed by an incoming selector value,“tuple.” As shown, the selector value is supplied to a LUT selector 434which asserts one of N enable signals (i.e., according to the value ofthe LUT selector) to enable one of the LUTs 435 ₀-435 _(N-1) to respondto the incoming “raw” ASP value by outputting an image pixel valuecorresponding to the point on the LUT-curve indexed by the ASP value.The enable signals are also supplied to a multiplexer 436 (or otherselector circuit) to pass the image pixel output from the enabled LUT todownstream logic or I/O circuitry.

Reflecting on the various embodiments of integrated-circuit imagesensors disclosed herein, it should be noted that the integrated-circuitimage sensors may, in all cases, be implemented on a single die or bymultiple dies (e.g., in a die-stack and/or side-by-side arrangement asin a system-on-chip, system-in-package, multi-die package, multi-chipmodule, package-on-package, package-in-package, and so forth). Forexample, some or all of the various logic functions (e.g., look-uptables, counter circuits, buffer circuits, etc.) may be implemented on adifferent die than the die bearing light-sensitive pixels, with thedifferent dies being wire-bonded, cabled, interconnected by TSVs orotherwise coupled to one another to form the final integrated-circuitimage sensor.

FIGS. 15A and 15B show an embodiment of a circuit and exemplaryoperation of a pixel 441 and a sense amplifier 451 as an example of anembodiment using a 4T-pixel with correlated double sampling (CDS). Tosupport CDS operation, pixel 441 has a floating diffusion (shown at node“FD” and sense amplifier 151 includes a sample-and-hold circuit 457(shown in alternative exemplary implementations at 456 and 458). Toreset pixel 441, an operation shown at 481 of FIG. 15B, signals Resetand BPR_(TG) are asserted concurrently, switching on transistors 442 and445 to charge photodiode 448. After a light accumulation interval 482,Reset and BPR_(SEL) are asserted while BPR_(TG) is not asserted. Thisoperation, which is shown at 483 of FIG. 15B, opens transistors 443 and444 to reset the floating diffusion. At 485 of FIG. 15B, signal LR isasserted to switch on transistor 461 within sample-and-hold circuit 457,thereby enabling the charge level at node FD (i.e., floating diffusionreset charge) to be stored in capacitive element CR. Thereafter, at 487of FIG. 15B, signal BPR_(SEL) is asserted together with BPR_(TG).BPR_(TG) moves the charge from the photodiode onto node FD. Concurrentassertion of signal LS switches on transistor 463 within sample-and-holdcircuit 457, thereby enabling storage a voltage representative of thephotodiode charge level within capacitive element CS. Lastly, at thestart of read-out/reset interval 489 (FIG. 15B), signal EC is assertedswitch on transistors 465 and 466 of sample-and-hold circuit 457,delivering the reset charge on CR and the photodiode charge on CS torespective inputs of a differential amplifier 468 and thus enabling thedifference between the reset charge and photodiode charge to be sensedin the sense-amplifier 451 (i.e., applied to the gate of transistor 216for differential sensing with respect to the threshold applied to thegate of transistor 171). After read-out/reset is completed, signal HiZmay be raised as shown at the tail end of interval 489 to switchinverter-enable transistor 456 to a non-conducting state, therebypowering off the inverter formed by transistors 176 and 177 andrendering the sense amplifier output (i.e., “Data” output of inverterformed by transistors 176, 177) to a high impedance state.

FIG. 16 illustrates an alternative embodiment of a binary pixel imagesensor 501 having a shared-counter architecture. In the implementationshown, image pixels (which are assumed for present purposes to becoextensive with algorithmic super pixels) are organized in “pixel mats”521, with each pixel mat formed by a group of image pixels that share arespective counter 523. Counters 523 (which may be viewed as adistributed implementation of the accumulator logic 157 of FIG. 1) serveto count pixel values obtained by sampling operations within thecorresponding pixel mats, with count values corresponding to respectiveimage pixels being provided to PHY 509 for output to an external memory503. Although each counter 523 in the sensor/counter array 507 isdepicted as being coupled to the PHY via a respective bit line 525,counters may share bit lines (e.g., all or some subset of counterswithin a given column may share a bit line).

Still referring to FIG. 16, sequencing logic 511 is provided to controlsampling operations within rows of image pixels and counting operationswithin the counters for respective image pixel mats. Control logic 515is provided to generate control and timing signals applied within thesequencing logic and PHY during image capture operations. In oneembodiment, for example, control logic 515 outputs a sampling clocksignal and transmit clock signal to the sequencing logic and PHY,respectively, with each clock signal establishing a separate clockdomain within image sensor IC 501. As discussed below, this arrangementenables a relatively steady data flow (in the transmit clock domain)from image sensor IC 501 to memory IC 503, despite a potentially morebursty internal data transfer (in the sampling clock domain) from thepixel mat counters to the PHY.

Referring to detail view 530, each image pixel mat is assumed to includefour image pixels (IP1-IP4), with each image pixel including a set offour binary pixels (i.e., image pixel IP1 includes binary pixels 1,1;1,2; 1,3 and 1,4). Accordingly, each image pixel value has a spatialoversampling factor (s) of 4, with the individual binary pixel samplesbeing accumulated within the corresponding pixel mat counter 523 beforebeing forwarded to PHY 509. Image pixels may also be temporallyoversampled as discussed above, with the results of iteratively executedspatial oversampling sequences being combined to form an s*n oversampledimage pixel value, where ‘*’ denotes multiplication and ‘n’ is theiteration count and thus the temporal oversampling factor.

FIG. 17 illustrates a spatial/temporal oversampling sequence withrespect to image pixel 1 (IP1) of the image pixel mat shown in detailview 530 of FIG. 16. As shown, each of the binary pixels, BP_(i,j),within IP1 is sampled in succession in a first spatial oversamplingsequence to yield sample values tallied (or accumulated or counted)within the corresponding pixel mat counter. This spatial-oversamplingsequence (i.e., sampling all the binary pixels of an image pixel andtallying the sample values in a pixel mat counter) is representedsymbolically by symbol 540. In the example shown, a second IP1 spatialoversampling sequence follows immediately after the first (i.e., noother image pixels in the same mat are sampled between the two IP1spatial-oversampling sequences) to effect 2× temporal oversampling(i.e., n=2). Generalizing the spatial oversampling factor and temporaloversampling factor as ranging from 1 to s and 1 to n, respectively (‘s’and ‘n’ representing maximum practicable limits), then the count valueaccumulated at the conclusion of any n successive spatial oversamplingsequences, may have any value from 0 to s*n, a range bearing on the sizeof the pixel mat counter (like the pixels themselves, however, the pixelmat counter may be sized to saturate, in the case of the counter at acount value lower than s*n).

To enable a counter to be shared among the constituent pixels of a mat,a count value accumulated for a given image pixel (i.e., after one ormore successive spatial-oversampling sequences) is transferred to memory(e.g., memory IC 503 of FIG. 16) and the counter is reset prior toaccumulating a count value for another image pixel (in an alternateembodiment, the counters can be allowed to “wrap” without reset, with aknown counter usage sequence being used to calculate differential countsbetween counter samplings). FIG. 18 illustrates this counttransfer/reset operation, with the output of the pixel mat counter beingtransferred to memory and the counter reset after each n successivespatial oversampling sequences with respect to a given image pixel(i.e., sequences in which no other image pixel within the same mat issampled). In an implementation in which n*s constitutes the total numberof image pixel samples acquired during a given exposure interval, theimage-pixel sequencing operation shown in FIG. 18 establishes a minimumuniform interval between each of the samples of a given binary pixel, atime corresponding to the spatial oversampling factor, s, times theminimum time between any two pixel sampling operations, t_(min) (i.e.,the BP_(i,j) sampling interval as shown in FIG. 17).

In an implementation that repeats the FIG. 18 sampling sequence multiple(‘r’) times in each exposure interval, several interestingcharacteristics arise. First, the sampling interval with respect to agiven binary pixel (or image pixel as a whole) becomes non-uniform, asthe s*t_(min) interval between successive samples of the same binarypixel during an uninterrupted image-pixel sampling sequence is followedby a much longer s*n*(m−1)*t_(min) interval (‘m’ being the number ofimage pixels per pixel mat) between the final and initial pixel samplesin successive iterations of the overarching sampling loop. Additionally,instead of forming the total value of a finalized image pixel to beapplied in an image reconstruction operation, the value output from agiven pixel mat counter following each uninterrupted image-pixelsampling sequence is but one of ‘r’ such values that collectively formthe total value of the image pixel samples. In effect, each image pixelsample-counting operation (i.e., counting samples during anuninterrupted image-pixel sampling sequence) yields a partialaccumulation of a larger total value. As discussed below, in a number ofembodiments such partial results are stored individually in memory andthen, during a subsequent reconstruction operation, retrieved andcombined with one another (e.g., added) to form the finalized imagepixel value. Comparing a uniform-interval sampling sequence (e.g.,effected by executing s*n*r samples of each image pixel of a given pixelmat in uninterrupted succession) with a non-uniform sampling sequence inwhich a group of s*n samples is acquired for each image pixel one afteranother in each of ‘r’ loops through the pixel mat, a number of benefitsof the latter approach become evident. First, because the maximum countachieved for a given image pixel is limited to the number of samplesacquired in one loop through the sampling pattern, the size of the pixelmat counter may be reduced by a factor of log₂(r) relative to theuniform-interval approach (i.e., assuming the same total number ofsamples in either approach)—a substantial reduction in counter circuitarea when a significant temporal oversampling factor (n*r) is employed.Additionally, as discussed above, mixing longer and shorter samplingintervals within the same exposure time may extend the effective dynamicrange of the image sensor. More specifically, given that the totalnumber of sampling operations that may be performed during a givenexposure interval is constrained by the number of pixels to be sampledand availability of shared counter/sense-amplifier resources, it followsthat, by acquiring a modest number of highest-resolution samples (i.e.,“fine samples”) of a given image pixel during a fraction of the exposureinterval and acquiring less frequent samples (“coarse samples”) for theremainder of the exposure interval, time and shared resources are freedto yield a potentially higher effective sampling resolution thanpossible in a uniform-interval scheme.

FIG. 19 illustrates an exemplary pixel sampling sequence in which eachimage pixel within a pixel mat is sampled with three different samplingintervals over the course of an exposure time, including a number offine sampling intervals (i.e., shortest time interval or highestresolution) and progressively extended first and second coarse samplingintervals. More specifically, referring to image pixel IP1, fourspatial-oversampling sequences are executed in succession followingrespective fine sampling intervals, with the net result beingaccumulated within the pixel mat counter and output from the counterfollowing the fourth spatial-oversampling sequence (i.e., as indicatedby transfer arrow 550). The ensuing six spatial-oversampling sequencesare directed to other image pixels within the same pixel mat beforeimage pixel IP1 is spatially oversampled again to yield a first coarsesample. Through this interleaved pixel sampling arrangement, a coarsesampling interval seven times the fine sampling interval is achievedwith respect to IP1 (i.e., “Coarse 1”), while at the same time freeingresources to finely and coarsely sample other image pixels within thesame pixel mat. After the first coarse sample is output from the pixelmat counter, the counter is freed to accumulate coarse and fine samplecounts for image pixels for an even longer second coarse samplinginterval (“Coarse 2”). Altogether, over an interval long enough topermit 24 spatial-oversampling sequences within the pixel mat, sixspatial over-sampling sequences are applied to each of the image pixels,including four in succession to the same image pixel (establishing thefine sampling intervals) and two interleaved with sampling operationsdirected to other image pixels. Each transition in the image pixelselected for sampling is preceded by a sample-count transfer (andcounter reset), so that a total of twelve sample-count transfers occursover the course of the 24 spatial-oversampling sequences shown, thusyielding a relatively bursty sample transfer with respect to a givenimage pixel mat. That is the transfers are bunched in groups betweenfine-sampling sequences, thus consuming internal and external datatransfer resources in bursts.

While the sequence shown in FIG. 19 may be executed iteratively toobtain the desired number of coarse and fine samples for each imagepixel, the pattern shown may also be extended to include even longercoarse intervals, further freeing shared data transfer resources andthus enabling time-multiplexed application of those resources todifferent pixel mats. More specifically, by time-staggering thesample-sequencing of image pixel mats that share a common resource(e.g., internal bit line, external data link and/or associatedcircuitry) to align data transfer bursts from one pixel mat with anon-burst interval in another, a relatively steady data transfer profilemay be obtained despite the bursty internal data transfer with respectto individual image pixels. FIG. 20 illustrates an exemplary samplesequencing that yields a levelized PHY output stream. As shown, samplingoperations that yield bursty internal data transfer are staggered fromimage pixel to image pixel within a pixel mat (i.e., from image pixelIP1 to IP2 to IP3 within pixel mat row 1 (M₁)) and from pixel mat topixel mat within a bit-line sharing column of pixel mats (i.e., from M₁to M₂ to M_(J)). In the example shown, fine sampling operations areshifted progressively later within the exposure interval as the pixelmat row index increases, with the image pixels of the final pixel matrow (M_(J)) exhibiting a sampling profile that is substantially thereverse of that for the first pixel mat row (M₁). That is, whereas theimage pixel sampling operations within the first row of pixel matstrends from fine to progressively more coarse sampling intervals, thesampling profile in the final row of pixel mats trends in the reversedirection from coarse to progressively finer sampling intervals. Inalterative embodiments, other mat-to-mat changes in sampling progressionare possible, including sampling progressions that are non-monotonic(i.e., not limited to increasing coarseness or decreasing coarseness)across all or most pixel mat rows.

Several variations on the basic sampling sequence can also be exploited.For longer overall frame times, the basic sequence can be lengthened bychanging the pixel clock, or changing the number of clock edges betweenpixel samples. Alternately, the basic sequence can be repeated multipletimes for a longer overall frame time (resulting in more data, butpossible reducing image noise). Also, variations of the basic sequence(with different base duration) can be serially concatenated during agiven frame.

FIG. 21 illustrates an exemplary storage of fractional image pixel countvalues within memory 503. In the example shown, the image count valuesare dispersed within one or more storage rows of the memory component(i.e., depicted as a single logical storage row 570 that could encompassmultiple physical storage rows) according to their acquisition timewithin a given exposure interval. Thus, count values corresponding togroups of fine samples and individual coarse samples of image pixelM_(1,1)IP₁ (i.e., image pixel IP1 of pixel mat row 1, column 1) arestored at a sequence of ascending addresses that are increasingly spacedapart (i.e., according to the progressively longer time betweensamples), while count values corresponding to groups of fine samples andindividual coarse samples of image pixel M_(J,K)IP₄ (i.e., image pixelIP4 of pixel mat row J, column K) are stored at a sequence of memorylocations that draw increasingly closer together with ascending memoryaddress. During image reconstruction, a reconstruction processor (whichmay be disposed on the same or different IC from the image sensor and/ormemory) retrieves fractional count values stored at dispersed storagelocations and sums or otherwise combines the fractional counts toproduce a finalized image pixel sample. The image pixel sample itselfmay thereafter be revised through further image processing or used as anindex (i.e., lookup value) to lookup another value representative of thelight intensity field detected by the subject binary pixels (e.g., asdiscussed in reference to FIG. 14). In an embodiment where the countersare allowed to wrap and are not reset after a count value is read, thereconstruction processor retrieves two fractional count values anddifferences them before summing or combining to produce a finalizedimage pixel sample.

FIG. 22 illustrates an alternative image sensor architecture 600 inwhich two rows of image pixel mats are operated in parallel, with eachpair of co-operated image pixel mats forming a “dmat” 615 (double mat).To avoid resource conflicts, separate bit lines 620 are coupled to theindividual pixel mats of each dmat, thus doubling the total number ofbit lines in the image sensor array. The return for the additional bits,is a halved number of control signals from sequencing logic 607 (i.e.,as only one set of row control signals is required for each two rows ofimage pixel mats) and halved image scan time. In the architecture shown,sense amplifiers and/or counters 603 are disposed at the periphery ofpixel array 601 adjacent PHY 605 (i.e., coupled to respective columns ofdmats via bit lines and sequenced in response to signals from ControlLogic 609) instead of being dispersed among the image pixels. In such anarchitecture, it may be possible to share a single pair of countersacross an entire column of image pixels, in effect, establishing asingle dmat per column of image pixels.

FIG. 23 illustrates a stacked-die implementation of an image sensor thatmay be used to implement the generalized image sensors of FIGS. 1, 16and 22. As shown, a pixel array is implemented in a first IC die 631(the “pixel array” die), while image pixel sense amplifiers and pixelmat counters are disposed on a second IC die, referred to herein as the“logic” die or sense-amp/counter die, beneath or otherwise in physicalproximity to corresponding pixels of a pixel mat. Thus the constituentpixels of a pixel mat 635 are disposed in the pixel array dieimmediately above (or adjacent to) the logic die implementation of senseamplifier(s) for the constituent image pixels of the pixel mat and pixelmat counter (collectively 637). In one embodiment, the pixel array dieis fabricated for backside illumination (i.e., ground, lapped orotherwise prepared in a manner that permits light detection through thebody of the substrate), thus permitting metal layers, bond pads, viasand/or other interconnect structures to be sandwiched between dies 631and 633 (i.e., fabricated on the side of the pixel array die facing thelogic die). The die-to-die interconnects shown conceptually at 639 maybe implemented by wire bonds, cables, through-silicon vias (TSVs), orany other practicable chip-to-chip interconnection structures.

FIG. 24 illustrates an example of the distinct pixel (sampling) clockand transmit clock domains described in reference to FIG. 1. As shown,the pixel clock (“Pix-Clk”) oscillates at a frequency corresponding tothe requisite pixel sampling rate within the image sensor (in this case,triggering a pixel sampling operation at every falling pixel-clock edgeduring a given image capture operation), while the transmit clock (“TxClk”) oscillates at a frequency corresponding to the desired datatransmission rate of the physical signaling interface (PHY). By thisarrangement, pixel sampling rates may be scaled up or down according toapplication requirements (or selected operating modes) without changingthe PHY signaling rate, thus enabling the image sensor to be operatedusing commodity memory components as the storage element for fractionalpixel count values. In one embodiment, the pixel clock and transmitclock may be derived from (or instantiated by) a common clock source(generated on or off chip) and thus have different frequencies but arelatively stable phase relation. Alternatively, the pixel and transmitclocks may result from different clock sources that exhibit frequencyand/or phase drift relative to one another. In either case, domaincrossing logic may be provided between the accumulator logic and PHY(e.g., elements 107 and 109 of FIG. 1, elements 603 and 605 of FIG. 22,etc.) to effect image data transfer between the two clock domains,buffering data as necessary to accommodate skip points between pixelclock and transmit clock edges (i.e., buffer load and unload clockedges).

FIG. 25 illustrates an image system architecture 650 that may be used toimplement the image sensor, image data storage and image reconstructionfunctions described in reference to FIG. 1. In the embodiment shown, amemory component 653 (which may be two or more memory components) issplit into two different memory regions (“Mem0,” “Mem1”) which arealternately loaded with image data from image sensor 651. Thus, during afirst image frame period or exposure interval (i.e., “Frame 1”), pixelcount values accumulated within image sensor IC 651 are output via PHY657 in a sequence of memory write operations directed to memory region0, with each such write operation effected by transmission of control,address and data signals on chip-to-chip signaling path 652. As shown bythe empty transfer slots between successive memory write operationsduring Frame 1, the write operations do not consume the entire bandwidthof signaling path 652 (or the memory component signaling interface),leaving bandwidth for reconstruction processor 655 to retrieve storedimage data from memory component 653 even as further image data is beingstored. Accordingly, during a subsequent image frame period (“Frame 2”),pixel count values accumulated within image sensor IC 651 (i.e.,corresponding to another image capture within pixel array 659) areoutput in a sequence of memory write operations directed to memoryregion 1, while an interleaved sequence of memory read accesses isexecuted by reconstruction processor 655 to retrieve image data stored(during Frame 1) in memory region 0. In subsequent frame 3, the sameinterleaved sequence of memory writes and reads is executed, with thedestination and source regions of memory being alternated so that imagesensor IC 651 writes Frame 3 image data to memory region 0, while thereconstruction processor retrieves Frame 2 image data from memoryregion 1. This alternation between destination and source memory regionsmay continue indefinitely as additional image frames are captured. Also,while a single memory component 653 is divided into different logicalstorage sections (or physical storage sections as in two differentinternal memory banks or two different groups of internal memory banks)in the embodiment of FIG. 25, two or more memory components may also beprovided, with memory regions 0 and 1 being implemented in separatememory components or separate groups of memory components. Further,while reconstruction processor 655 is depicted as a dedicated IC withinarchitecture 650, the reconstruction processor may alternatively bedisposed on the same IC die as image sensor 651 and/or memory 653. Wheretwo or more dies are used to implement the architecture shown in FIG.25, the dies may be packaged as a unit, as in a three-dimensional IC,multi-chip module, multi-chip package, package in package, package onpackage and so forth.

It should be noted that the various circuits disclosed herein can bedescribed using computer aided design tools and expressed (orrepresented), as data and/or instructions embodied in variouscomputer-readable media, in terms of their behavioral, registertransfer, logic component, transistor, layout geometries, and/or othercharacteristics. Formats of files and other objects in which suchcircuit expressions can be implemented include, but are not limited to,formats supporting behavioral languages such as C, Verilog, and VHDL,formats supporting register level description languages like RTL, andformats supporting geometry description languages such as GDSII, GDSIII,GDSIV, CIF, MEBES and any other suitable formats and languages.Computer-readable media in which such formatted data and/or instructionscan be embodied include, but are not limited to, computer storage mediain various forms (e.g., optical, magnetic or semiconductor storagemedia, whether independently distributed in that manner, or stored “insitu” in an operating system).

When received within a computer system via one or more computer-readablemedia, such data and/or instruction-based expressions of the abovedescribed circuits can be processed by a processing entity (e.g., one ormore processors) within the computer system in conjunction withexecution of one or more other computer programs including, withoutlimitation, net-list generation programs, place and route programs andthe like, to generate a representation or image of a physicalmanifestation of such circuits. Such representation or image canthereafter be used in device fabrication, for example, by enablinggeneration of one or more masks that are used to form various componentsof the circuits in a device fabrication process.

In the foregoing description and in the accompanying drawings, specificterminology and drawing symbols have been set forth to provide athorough understanding of the disclosed embodiments. In some instances,the terminology and symbols may imply specific details that are notrequired to practice those embodiments. For example, any of the specificnumbers of bits, signal path widths, signaling or operating frequencies,component circuits or devices and the like can be different from thosedescribed above in alternative embodiments. Additionally, links or otherinterconnection between integrated circuit devices or internal circuitelements or blocks may be shown as buses or as single signal lines. Eachof the buses can alternatively be a single signal line, and each of thesingle signal lines can alternatively be buses. Signals and signalinglinks, however shown or described, can be single-ended or differential.A signal driving circuit is said to “output” a signal to a signalreceiving circuit when the signal driving circuit asserts (or deasserts,if explicitly stated or indicated by context) the signal on a signalline coupled between the signal driving and signal receiving circuits.The term “coupled” is used herein to express a direct connection as wellas a connection through one or more intervening circuits or structures.Integrated circuit device “programming” can include, for example andwithout limitation, loading a control value into a register or otherstorage circuit within the integrated circuit device in response to ahost instruction (and thus controlling an operational aspect of thedevice and/or establishing a device configuration) or through a one-timeprogramming operation (e.g., blowing fuses within a configurationcircuit during device production), and/or connecting one or moreselected pins or other contact structures of the device to referencevoltage lines (also referred to as strapping) to establish a particulardevice configuration or operation aspect of the device. The terms“exemplary” and “embodiment” are used to express an example, not apreference or requirement. Also, the terms “may” and “can” are usedinterchangeably to denote optional (permissible) subject matter. Theabsence of either term should not be construed as meaning that a givenfeature or technique is required.

The section headings in the above detailed description have beenprovided for convenience of reference only and in no way define, limit,construe or describe the scope or extent of the corresponding sectionsor any of the embodiments presented herein. Also, various modificationsand changes can be made to the embodiments presented herein withoutdeparting from the broader spirit and scope of the disclosure. Forexample, features or aspects of any of the embodiments can be applied,at least where practicable, in combination with any other of theembodiments or in place of counterpart features or aspects thereof.Accordingly, the specification and drawings are to be regarded in anillustrative rather than a restrictive sense.

What is claimed is:
 1. A method of operation with an image sensor, themethod comprising: accumulating photocharge within a two-dimensionalarray of pixel groups disposed in a first integrated circuit die, eachof the pixel groups including a two-dimensional array of pixels;outputting signals representative of the photocharge accumulated withinthe two-dimensional array of pixel groups to a two-dimensional array ofsense-amp/counter circuits disposed in a second integrated circuit die,the second integrated circuit die being coupled in a stackedconfiguration with the first integrated circuit die to align and enableelectrical coupling between each of the sense/amp counter circuits and arespective, counterpart pixel group within the two-dimensional array ofpixel groups; generating, within each of the sense-amp/counter circuitsand in accordance with the signals representative of photochargeaccumulated, digital values representative of photocharge accumulatedwithin respective pixels of the two-dimensional array of pixels includedwithin the counterpart pixel group; and generating, within each of thesense-amp/counter circuits, a count value that constitutes a sum of thedigital values representative of photocharge accumulated within therespective pixels of the two-dimensional array of pixels included withinthe counterpart pixel group.
 2. The method of claim 1 wherein outputtingsignals representative of the photocharge accumulated within thetwo-dimensional array of pixel groups comprises outputting signals fromeach of the pixel groups to each of the sense-amp/counter circuitssimultaneously.
 3. The method of claim 2 wherein outputting signals fromeach of the pixel groups to each of the sense/amp counter circuitssimultaneously comprises, for each one of the pixel groups, outputting asequence of signals that correspond to photocharge accumulated withinrespective pixels of the two-dimensional array of pixels included withinthe one of the pixel groups.
 4. The method of claim 3 whereingenerating, within each of the sense-amp/counter circuits, digitalvalues representative of photocharge accumulated within pixels of thetwo-dimensional array of pixels comprises generating a respectivedigital value for each signal of the sequence of signals.
 5. The methodof claim 1 wherein generating, within each of the sense-amp/countercircuits, digital values representative of photocharge accumulatedwithin pixels of the two-dimensional array of pixels comprisesgenerating a respective binary value representative of photochargeaccumulated within each pixel of the two-dimensional array of pixelsincluded within the counterpart pixel group.
 6. The method of claim 5wherein generating the respective binary value representativephotocharge accumulated within each pixel comprises generating, for eachpixel, a single-bit digital value having either a first logic state or asecond logic state according to whether the photocharge accumulatedwithin the pixel exceeds a predetermined threshold.
 7. The method of 1wherein accumulating photocharge within the two dimensional array ofpixel groups, outputting signals representative of the photochargeaccumulated and generating digital values representative of thephotocharge accumulated constitutes a first exposure and read-out of thetwo-dimensional array of pixel groups, and wherein the method furthercomprises executing additional exposures and corresponding read-outs ofthe two-dimensional array of pixel groups in which each read-out isfollowed by cumulative increase in the count value within each of thesense-amp/counter circuits in accordance with the digital valuesrepresentative of photocharge accumulated during the correspondingexposure.
 8. The method of claim 7 further comprising resetting eachpixel within each pixel group of the two-dimensional array of pixelgroups following each read-out.
 9. The method of claim 7 wherein thefirst exposure and the additional exposures transpire over a frameinterval, the method further comprising outputting, at the conclusion ofthe frame interval and as a constituent pixel value within a digitalimage corresponding to the frame interval, the count value from each ofthe sense/amp counter circuits.
 10. The method of claim 9 wherein thecount value output from each of the sense/amp counter circuits at theconclusion of the frame interval has a respective value between 0 andN*M, where N is the number of pixels within each two-dimensional arrayof pixels, M is number of exposures that transpire within the frameinterval, and ‘*’ designates multiplication.
 11. An image sensorcomprising: a two-dimensional array of pixel groups disposed in a firstintegrated circuit die, each of the pixel groups including atwo-dimensional array of pixels to accumulate photocharge; atwo-dimensional array of sense-amp/counter circuits disposed in a secondintegrated circuit die that is coupled in a stacked configuration withthe first integrated circuit die to align and effect electrical couplingbetween each of the sense/amp counter circuits and a respective,counterpart pixel group within the two-dimensional array of pixelgroups; and control circuitry disposed within at least one of the firstand second integrated circuit dies to output control signals thatenable: the two-dimensional array of pixel groups to output signalsrepresentative of the photocharge accumulated within constituent pixelsthereof to the two-dimensional array of sense-amp/counter circuitsdisposed in the second integrated circuit die; generation, within eachof the sense-amp/counter circuits and in accordance with the signalsrepresentative of photocharge accumulated, of digital valuesrepresentative of photocharge accumulated within respective pixels ofthe two-dimensional array of pixels included within the counterpartpixel group; and generation, within each of the sense-amp/countercircuits, of a count value that constitutes a sum of the digital valuesrepresentative of photocharge accumulated within the respective pixelsof the two-dimensional array of pixels included within the counterpartpixel group.
 12. The image sensor of claim 11 wherein the controlcircuitry to output control signals that enable the two-dimensionalarray of pixels to output signals representative of the photochargeaccumulated within the two-dimensional array of pixel groups comprisescircuitry to enable the two-dimensional array of pixels to outputsignals from each of the pixel groups to each of the sense-amp/countercircuits simultaneously.
 13. The image sensor of claim 12 wherein thecontrol circuitry to output control signals that enable thetwo-dimensional array of pixels to output signals from each of the pixelgroups to each of the sense/amp counter circuits simultaneouslycomprises circuitry to enable the two-dimensional array of pixels tooutput, for each one of the pixel groups, a sequence of signals thatcorrespond to photocharge accumulated within respective pixels of thetwo-dimensional array of pixels included within the one of the pixelgroups.
 14. The image sensor of claim 13 wherein the control circuitryto output control signals that enable generation, within each of thesense-amp/counter circuits, of digital values representative ofphotocharge accumulated within pixels of the two-dimensional array ofpixels comprises circuitry to enable generation, within each of thesense-amp/counter circuits, of a respective digital value for eachsignal of the sequence of signals.
 15. The image sensor of claim 11wherein the control circuitry to output control signals that enablegeneration, within each of the sense-amp/counter circuits, of digitalvalues representative of photocharge accumulated within pixels of thetwo-dimensional array of pixels comprises circuitry to enablegeneration, within each of the sense-amp/counter circuits, of arespective binary value representative of photocharge accumulated withineach pixel of the two-dimensional array of pixels included within thecounterpart pixel group.
 16. The image sensor of claim 15 wherein thecircuitry to enable generation of the respective binary valuerepresentative photocharge accumulated within each pixel comprisescircuitry to enable generation, for each pixel, of a single-bit digitalvalue having either a first logic state or a second logic stateaccording to whether the photocharge accumulated within the pixelexceeds a predetermined threshold.
 17. The image sensor of 11 whereinthe control circuitry operates iteratively to enable generation of thedigital values representative of the photocharge accumulated within thetwo-dimensional array of pixel groups following each of a plurality ofexposure intervals, and to enable cumulative increase in the count valuewithin each of the sense-amp/counter circuits in accordance with thedigital values representative of photocharge accumulated during thecorresponding exposure interval.
 18. The image sensor of claim 17wherein the signals output by the control circuitry act to reset eachpixel within each pixel group of the two-dimensional array of pixelgroups following each exposure interval.
 19. The image sensor of claim17 wherein the plurality of exposure intervals transpire over a frameinterval, and wherein the signals output by the control circuitryfurther enable the each of the sense-amp/counter circuits to output, atthe conclusion of the frame interval and as a constituent pixel valuewithin a digital image corresponding to the frame interval, the countvalue accumulated therein, and wherein the count value output from eachof the sense/amp counter circuits at the conclusion of the frameinterval has a respective value between 0 and N*M, where N is the numberof pixels within each two-dimensional array of pixels, M is number ofexposures that transpire within the frame interval, and ‘*’ designatesmultiplication.
 20. An image sensor comprising: a two-dimensional arrayof pixel groups disposed in a first integrated circuit die, each of thepixel groups including a two-dimensional array of pixels to accumulatephotocharge; a two-dimensional array of sense-amp/counter circuitsdisposed in a second integrated circuit die that is coupled in a stackedconfiguration with the first integrated circuit die to align and effectelectrical coupling between each of the sense/amp counter circuits and arespective, counterpart pixel group within the two-dimensional array ofpixel groups; and means for outputting control signals that enable: thetwo-dimensional array of pixel groups to output signals representativeof the photocharge accumulated within constituent pixels thereof to thetwo-dimensional array of sense-amp/counter circuits disposed in thesecond integrated circuit die; generation, within each of thesense-amp/counter circuits and in accordance with the signalsrepresentative of photocharge accumulated, of digital valuesrepresentative of photocharge accumulated within respective pixels ofthe two-dimensional array of pixels included within the counterpartpixel group; and generation, within each of the sense-amp/countercircuits, of a count value that constitutes a sum of the digital valuesrepresentative of photocharge accumulated within the respective pixelsof the two-dimensional array of pixels included within the counterpartpixel group.